Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update reference tests to Cancun #6054

Merged
merged 7 commits into from
Oct 25, 2023
Merged

Update reference tests to Cancun #6054

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
242d6a7
Update reference tests to Cancun
shemnon Oct 18, 2023
26bfc1f
update the subrepo
shemnon Oct 18, 2023
11fbbf8
update the EIP-6110 acceptance tests
shemnon Oct 19, 2023
9940f26
merge
shemnon Oct 19, 2023
731db59
update to develop
shemnon Oct 23, 2023
a61a63b
merge
shemnon Oct 23, 2023
52bdcb4
Merge branch 'main' into reftests_pre_cancun
macfarla Oct 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 1 addition & 3 deletions ...core/src/main/java/org/hyperledger/besu/ethereum/mainnet/ParentBeaconBlockRootHelper.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
import org.hyperledger.besu.evm.account.MutableAccount;
import org.hyperledger.besu.evm.worldstate.WorldUpdater;

import com.google.common.primitives.Longs;
import org.apache.tuweni.bytes.Bytes;
import org.apache.tuweni.bytes.Bytes32;
import org.apache.tuweni.units.bigints.UInt256;
Expand Down Expand Up @@ -49,8 +48,7 @@ static void storeParentBeaconBlockRoot(
final UInt256 timestampIndex = UInt256.valueOf(timestampReduced);
final UInt256 rootIndex = UInt256.valueOf(timestampExtended);

account.setStorageValue(
timestampIndex, UInt256.fromBytes(Bytes.of(Longs.toByteArray(timestamp))));
account.setStorageValue(timestampIndex, UInt256.fromBytes(Bytes.ofUnsignedLong(timestamp)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a conflict here with the change made by @jflo here https://github.com/hyperledger/besu/pull/6046/files#diff-61db834b59eae5ce5c438462505de1add8fa244deda830742060d15f668a9806

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same change but with better APIs. The #6046 change was to store values as unsigned longs. The old way created multiple Bytes objects, which causes bad performance (there was a prior optimization addressing deeply wrapped bytes). The use of Bytes.ofUnsignedLong skips the multiple object creations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ha! awesome, i hated how I did that.

account.setStorageValue(rootIndex, UInt256.fromBytes(root));
worldUpdater.commit();
}
Expand Down
8 changes: 8 additions & 0 deletions ethereum/evmtool/src/main/java/org/hyperledger/besu/evmtool/StateTestSubCommand.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ public class StateTestSubCommand implements Runnable {
description = "Limit execution to one value variable.")
private Integer valueIndex = null;

@Option(
names = {"--fork-index"},
description = "Limit execution to one fork.")
private String forkIndex = null;

@ParentCommand private final EvmToolCommand parentCommand;

// picocli does it magically
Expand Down Expand Up @@ -197,6 +202,9 @@ private void traceTestSpecs(final String test, final List<GeneralStateTestCaseEi
if (valueIndex != null && spec.getValueIndex() != valueIndex) {
continue;
}
if (forkIndex != null && !spec.getFork().equalsIgnoreCase(forkIndex)) {
continue;
}

final BlockHeader blockHeader = spec.getBlockHeader();
final Transaction transaction = spec.getTransaction();
Expand Down
1 change: 0 additions & 1 deletion ethereum/evmtool/src/main/java/org/hyperledger/besu/evmtool/T8nExecutor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,6 @@ static T8nResult runTest(

final BonsaiReferenceTestWorldState worldState =
(BonsaiReferenceTestWorldState) initialWorldState.copy();
worldState.disableRootHashVerification();

final ProtocolSchedule protocolSchedule = referenceTestProtocolSchedules.getByName(fork);
if (protocolSchedule == null) {
Expand Down
2 changes: 1 addition & 1 deletion ethereum/referencetests/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,7 +198,7 @@ tasks.register('validateReferenceTestSubmodule') {
description = "Checks that the reference tests submodule is not accidentally changed"
doLast {
def result = new ByteArrayOutputStream()
def expectedHash = '661356317ac6df52208d54187e692472a25a01f8'
def expectedHash = '02c1859633abbdf776956136931ea247485c9f90'
def submodulePath = java.nio.file.Path.of("${rootProject.projectDir}", "ethereum/referencetests/src/reference-test/external-resources").toAbsolutePath()
try {
exec {
Expand Down
1 change: 1 addition & 0 deletions ...in/java/org/hyperledger/besu/ethereum/referencetests/BlockchainReferenceTestCaseSpec.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,7 @@ public ParsedExtraData parseExtraData(final BlockHeader header) {
"expectExceptionHomestead",
"expectExceptionALL",
"hasBigInt",
"rlp_decoded",
"transactionSequence"
})
public static class CandidateBlock {
Expand Down
25 changes: 9 additions & 16 deletions ...main/java/org/hyperledger/besu/ethereum/referencetests/BonsaiReferenceTestWorldState.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,6 @@ public class BonsaiReferenceTestWorldState extends BonsaiWorldState
private final BonsaiReferenceTestWorldStateStorage refTestStorage;
private final BonsaiPreImageProxy preImageProxy;

private boolean disableRootHashVerification;

protected BonsaiReferenceTestWorldState(
final BonsaiReferenceTestWorldStateStorage worldStateStorage,
final CachedMerkleTrieLoader cachedMerkleTrieLoader,
Expand Down Expand Up @@ -78,20 +76,15 @@ public ReferenceTestWorldState copy() {
}

/**
* If the supplied header has a non-zero state root, verify. Else we assume that stateroot is an
* output instead of an input for this reference test and we bypass the state root check.
*
* <p>Besu reference-test style test cases should supply a stateroot to verify to prevent bonsai
* regressions.
* For reference tests world state root validation is handled in the harness, this stubs out the
* behavior to always pass.
*
* @param calculatedStateRoot state root calculated during bonsai persist step.
* @param header supplied reference test block header.
*/
@Override
protected void verifyWorldStateRoot(final Hash calculatedStateRoot, final BlockHeader header) {
if (!disableRootHashVerification) {
super.verifyWorldStateRoot(calculatedStateRoot, header);
}
// The test harness validates the root hash, no need to validate in-line for reference test
}

@JsonCreator
Expand Down Expand Up @@ -127,10 +120,6 @@ public Stream<StreamableAccount> streamAccounts(final Bytes32 startKeyHash, fina
return this.refTestStorage.streamAccounts(this, startKeyHash, limit);
}

public void disableRootHashVerification() {
disableRootHashVerification = true;
}

static class NoOpTrieLogManager implements TrieLogManager {
private final Subscribers<TrieLogEvent.TrieLogObserver> trieLogObservers = Subscribers.create();
private final TrieLogFactory trieLogFactory = new TrieLogFactoryImpl();
Expand All @@ -150,7 +139,9 @@ public void saveTrieLog(
public void addCachedLayer(
final BlockHeader blockHeader,
final Hash worldStateRootHash,
final BonsaiWorldState forWorldState) {}
final BonsaiWorldState forWorldState) {
// reference tests do not cache layers
}

@Override
public boolean containWorldStateStorage(final Hash blockHash) {
Expand Down Expand Up @@ -179,7 +170,9 @@ public long getMaxLayersToLoad() {
}

@Override
public void reset() {}
public void reset() {
// reference test world states are not re-used
}

@Override
public Optional<? extends TrieLog> getTrieLogLayer(final Hash blockHash) {
Expand Down
2 changes: 1 addition & 1 deletion ethereum/referencetests/src/reference-test/external-resources
Open in desktop
Submodule external-resources updated from 661356 to 02c185
7 changes: 0 additions & 7 deletions ...rc/reference-test/java/org/hyperledger/besu/ethereum/vm/BlockchainReferenceTestTools.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,6 @@ public class BlockchainReferenceTestTools {
// Perfectly valid test pre-merge.
params.ignore("UncleFromSideChain_(Merge|Shanghai|Cancun|Prague|Osaka|Bogota)");

// Reference Tests are old. Max blob count is 6.
params.ignore("blobhashListBounds5");
params.ignore("blockWithAllTransactionTypes");

// EIP-4788 is still in flux and the current fill is not against the final address
params.ignore("\\[Cancun\\]");

// EOF tests are written against an older version of the spec
params.ignore("/stEOF/");
}
Expand Down
7 changes: 0 additions & 7 deletions .../reference-test/java/org/hyperledger/besu/ethereum/vm/GeneralStateReferenceTestTools.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,8 @@
import org.hyperledger.besu.datatypes.Wei;
import org.hyperledger.besu.ethereum.core.BlockHeader;
import org.hyperledger.besu.ethereum.core.BlockHeaderBuilder;
import org.hyperledger.besu.ethereum.core.BlockHeaderFunctions;
import org.hyperledger.besu.ethereum.core.MutableWorldState;
import org.hyperledger.besu.ethereum.core.Transaction;
import org.hyperledger.besu.ethereum.mainnet.MainnetBlockHeaderFunctions;
import org.hyperledger.besu.ethereum.mainnet.MainnetTransactionProcessor;
import org.hyperledger.besu.ethereum.mainnet.ProtocolSpec;
import org.hyperledger.besu.ethereum.mainnet.TransactionValidationParams;
Expand All @@ -35,10 +33,8 @@
import org.hyperledger.besu.ethereum.referencetests.ReferenceTestProtocolSchedules;
import org.hyperledger.besu.ethereum.referencetests.ReferenceTestWorldState;
import org.hyperledger.besu.ethereum.rlp.RLP;
import org.hyperledger.besu.ethereum.worldstate.DefaultMutableWorldState;
import org.hyperledger.besu.evm.account.Account;
import org.hyperledger.besu.evm.log.Log;
import org.hyperledger.besu.evm.worldstate.WorldState;
import org.hyperledger.besu.evm.worldstate.WorldUpdater;
import org.hyperledger.besu.testutil.JsonTestParameters;

Expand Down Expand Up @@ -109,9 +105,6 @@ private static ProtocolSpec protocolSpec(final String name) {
params.ignore("CALLBlake2f_MaxRounds.*");
params.ignore("loopMul-.*");

// Reference Tests are old. Max blob count is 6.
params.ignore("blobhashListBounds5");

// EOF tests are written against an older version of the spec
params.ignore("/stEOF/");
}
Expand Down